Some video processing tasks, such as noise reduction for TV video signals, de-interlacing (or interlace to progressive conversion), and 3D combing for Y/C separation, usually involve operations on pixels in consecutive pictures (frames or fields). For example, in analog noise reduction (NR), the pixel in consideration (denoted as the current pixel) in the current picture is filtered together with certain pixels in the previous pictures and/or next pictures (of the same parity) in the temporal order. The next pictures are usually unprocessed. Some or all the previous pictures can be previously processed pictures, in which case the filtering is IIR in nature. If none of the previous pictures is processed yet, the filtering is FIR in nature. The filtered result is usually blended with the current un-processed pixel to get the output pixel value. The decision on how much each part contributes to the final output is critical in giving good results. Some algorithms make the decision on how the filtered result is blended with the unfiltered pixel according to motion information. The principle is to have the filtered result to contribute a larger proportion to the blend when there is little or no motion at the pixel position, and contribute smaller proportion to the blend when there is a larger amount of motion at the pixel position. Inappropriate blending would result in artifacts of motion blur, motion trails, jitters, wobblings, or inadequate noise removal. The measure of the amount of motion can be done in different forms. The mean absolute difference (MAD) between neighboring pictures over a neighborhood of the pixel in consideration has been widely used.
In motion-compensation-based temporal filtering (MCTF) for noise reduction, similar principles apply. The locations of corresponding pixels in different pictures should be aligned based on the motion vectors (MVs) obtained through a motion estimation process when the MVs represent meaningful motion of the picture content. Prior to motion compensation, at least two of the corresponding pixels in different pictures may have different coordinates within the respective pictures.
In de-interlacing, each field picture is converted to a frame picture. An absent pixel between two vertically neighboring existing pixels is generated. If there is no motion between the current field and its neighboring field(s), the pixel(s) that co-located with the absent pixel can be used to generate the absent pixel. This operation is called the temporal interpolation (TI), and is also traditionally called weave. Weave has the benefit of keeping the generated frame picture sharp as it utilizes original pixel values to increase picture resolution. In some instances, use of TI may result in bad weaves, which would cause a type of video distortion referred to as false motion effect.
If there is significant motion of the picture content, the neighboring existing pixels in the same field can be used to generate the absent pixel. This operation is called spatial interpolation (SI). SI is also traditionally called a bob. Bobbing is bad-weave free, but may result in video frames that look soft or blurry. When the motion is measured as a fuzzy indicator, the weave and bob results are blended, according to a blending factor derived from the motion metric.
Similar principles can be applied to 3D combing algorithms for Y/C (luma-chroma) separation of NTSC or PAL signals. In an NTSC signal, for example, the collocated pixels in two consecutive video frames are of opposite phase for the color sub-carrier. The color signals would cancel out and only the luma component would remain if the corresponding pixels were added together when there was no motion detected at a pixel position. The color would also mostly cancel out when there was little detected motion. When there is significant motion, this inter-frame operation cannot be reliably used to cancel out the chroma. Instead, intra-frame operations should be used. And blending of the results from the inter-frame operations and intra-frame operations can be based on a blending factor according to the motion metric.
The most traditional method for deriving the blending factor usually involves mapping the value of the motion metric to the value of the blending factor, linearly or piece-wise linearly. In many cases, the linear, or piece-wise linear, mapping lacks sensitivity to motion. This may result in slow reaction to increased motion in the picture content. The slow reaction may produce disturbing artifacts in processed video pictures such as motion blur, motion trails, motion aliasing or incorrect luma/chroma separation. Methods such as controlling the blending through linear mapping based on quantized values of the motion metric have also been proposed. Utilizing quantized values of the motion metric could also result in banding artifacts, in addition to other artifacts mentioned above.
Further limitations and disadvantages of conventional and traditional approaches will become apparent to one of skill in the art, through comparison of such systems with some aspects of the present invention as set forth in the remainder of the present application with reference to the drawings.